Device for monitoring the environment of a vehicle being parked

ABSTRACT

A device for monitoring the environment of a vehicle being parked, which is used for reliably detecting objects in the way, and which makes it possible to calculate parking maneuvers. The device includes at least one video camera having laterally arranged object-detection sensors, which cover areas that are not recorded by the video camera. In this manner, the driver can be supplied in a simple manner with warnings, even though a putative obstacle is not detectable in the video image.

FIELD OF THE INVENTION

The present invention relates to a device for monitoring the environmentof a vehicle being parked.

BACKGROUND INFORMATION

A conventional device for monitoring the environment of a vehicle beingparked is described in German Patent No. 43 36 288. In the devicedescribed, means are provided for swiveling a camera to expand thevision field of the camera.

SUMMARY OF THE INVENTION

In contrast, the device according to the present invention has theadvantage that on the basis of a simple, robust arrangement, reliablemonitoring of the vehicle environment can be assured. Movable parts thatare subject to wear are avoided, and the dead angles of the video cameraare included in the monitoring area. This allows the driver to receive adanger message in response to, for example, persons situated in the deadangle of the camera, and not only when questionable obstacles arelocated in the vision field of the camera. Therefore, there is aneconomic distribution of functions in detecting objects and reporting tothe driver between the object-detection sensors on the one hand, and thecamera on the other.

It is advantageous that the existence of objects outside of the visionfield of the camera can be represented separately via a display unit. Asa result, the attention of the driver is immediately guided in thedirection in which the danger threatens, although he is still unable todetect anything through the video image.

It is particularly advantageous to provide an object-detection unitwhich, for evaluating objects, processes both data from theobject-detection sensors as well as the video images. Combining aplurality of sensor information units in an overall display has theadvantage that the behavior of the system is always transparent becausethe driver can also check the information visually. The driver is notlimited to the few parameters that isolated object-detection sensorswould supply, such as the vehicle distance to the person behind. As aresult, it is possible at any time for the driver, in a simple fashion,to carry out monitoring and driving correction.

The visual presentation of the results in the image of the recordingvideo camera has the advantage that the driver can very effectivelysurvey the situation, since he is accustomed to steer the vehicle basedon visual information.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 depicts a top view of a vehicle having a monitoring deviceaccording to an embodiment of the present invention.

FIG. 2 depicts a vehicle having a further device according to anembodiment of the present invention.

FIG. 3 depicts a schematic representation of a further device accordingto an embodiment of the present invention.

FIG. 4 depicts a video display unit according to an embodiment of thepresent invention.

FIG. 5 depicts a further video display unit according to an embodimentof the present invention.

FIG. 6 depicts an exemplary parking scenario.

DETAILED DESCRIPTION

FIG. 1 depicts a top view of a vehicle 7. In the rear space of vehicle7, a video camera la disposed which monitors the rear area of thevehicle in a vision field 8 a. Mounted adjacent to video camera la areobject-detection sensors 9 a and 9 b, which are used to detect objectsin environment sectors 10 a and 10 b, respectively. These environmentsensors 10 a and 10 b directly adjoin vision field 8 a, but lie outsideof it. In addition, forward-directed object-detection sensors 9 c, 9 d,and 9 e are arranged in the area of the front bumpers of the motorvehicle, the sensors monitoring environment sectors 10 c, 10 d, and 10e, which are directed toward the front area.

Video camera 1 a, non-movable and fixedly installed at minimal expense,covers a prescribed vision field 8 a which can be displayed to thedriver by a video display unit arranged, for example, in the dashboardor the center console of the vehicle. Using object-detection sensors 9 aand 9 b, it is possible to detect objects that are approaching the areamonitored by the camera but that are not yet registered in the image.Detected objects of this kind can then be reported to the driver eventhough they cannot yet be detected in the camera image. The object canbe, for example, a child that is approaching the vehicle from the side,although the driver cannot see this child either in the displayed imageof the camera or in the rear view mirrors, because the child is locatedin the dead angle. The object-detection sensors are configured, forexample, as ultrasound, radar, video, or lidar sensors. Additionalsensors 9 c, 9 d, and 9 e can be used during parking maneuvers formeasuring at all times the distance to the vehicle parked ahead or fordetecting other (endangered) objects. If the distance to the forwardvehicle or another object is too small, a message can be transmitted.

The additional sensors can also be installed at any location in thevehicle to monitor the corresponding environment. For example, thetime-related vehicle area offers further installation possibilities, inwhich objects can be observed and appropriate warnings can be providedto the driver. If the object detection is carried out in a monocularfashion, economies can be achieved because only one camera is used.However, such methods have the disadvantage, in comparison to stereomethods, that they do not deliver very precise and dependable results.By combining monocular object detection with other sensors such asultrasound sensors, precision and reliability can be significantlyincreased.

FIG. 2 depicts vehicle 7, which has additionally installed video cameras1 b and 1 c, whose corresponding vision fields 8 b and 8 c cover thefront area of the vehicle. In this context, the cameras are mountedbetween object-detection sensors 9 c, 9 d, and 9 e, so that the threeaforementioned object-detection sensors, as a supplement to the cameras,monitor the peripheral areas of vision fields 8 b and 8 c. Furthermore,other object-detection sensors 9 f and 9 g are mounted in the rear partof the vehicle which have corresponding environment sectors 10 f and 10g that partially overlap vision field 8 a of the video camera 1 a,bringing these areas of vision field 8 a into the monitoring area insupplement to the video camera.

Providing additional object-detection sensors in the rear part of thevehicle results in increasing the reliability of object detection bygenerating data redundancy. If vehicle 7 is operated using a trailer,then, in the event that the trailer obscures the vision field or theenvironment sectors of the rear camera or of the rear object-detectionsensors, the trailer can be provided with an analogous monitoring device1 a, 9 a, 9 b, 9 f, 9 g, which is directed at the area behind thetrailer. Thus, vehicles in trailer operation can also support the driverduring parking maneuvers.

FIG. 3 depicts the schematic design of a device for monitoring theenvironment of a vehicle being parked, in which the totality of thevideo cameras is designated by reference numeral 1 and the totality ofthe object-detection sensors is designated by reference numeral 9. Thevideo cameras and the object-detection sensors are connected to acontrol unit 20. Control unit 20 has an image processing unit 2, whichprocesses the image data of video cameras 1. Image processing unit 2delivers preprocessed image data to an object-detection unit 30, whichprocesses both the preprocessed image data of the video cameras as wellas the signals of object-detection sensors 9. Detected objects or dangersituations can be reported to the driver via an acoustical display 6.The acoustical display is arranged in the passenger area of the vehicle.A superimposition unit 40 is connected to image processing unit 2 and toobject-detection unit 30. This superimposition unit 40 superimposesinformation supplied by object-detection unit 30 with respect todetected objects and the pre-processed image data from image processingunit 2 for representation in a video display unit 55 or 55′.Object-detection unit 30 is also connected to a maneuver calculatingunit 50, which, from the object data of object-detection unit 30 andexternally supplied parameters 4, for example the selected steeringwheel angle, calculates driving maneuvers and transmits this data tosuperimposition unit 40 for visual representation in the video displayunit. Maneuver calculating unit 50 is also connected to a control unit90 for the purpose of independently carrying out a driving maneuver.

In control unit 20, the images of the video cameras can be pre-processedusing image processing algorithms, and they can be displayed on a videodisplay unit 55 or 55′. In this context, the algorithms of the controlunit can also use vehicle parameters such as speed and the steeringangle of the steering wheel. In the video display unit, in addition tothe image content of the cameras, supplemental information such aswarnings about objects in the vehicle environment can also be displayed.The possibility also exists of generating warnings acoustically viaacoustical display 6. In this context, image processing unit 2 includesalgorithms for image preprocessing, such as noise suppression, imagerectification, or the like. The processed images may be combined withsupplemental image contents using superimposition unit 40 and would bedisplayed on the video display unit. Object-detection unit 30 receivesdata from the object-detection sensors and from the image processingunit. Detected objects are transmitted to superimposition unit 40 fordisplay in the video display unit and are also relayed to maneuvercalculating unit 50 for calculating maneuvers. In order to calculatemaneuvers, external parameters can be taken into consideration. Themaneuver calculating unit can prepare the calculated maneuvers suitablyfor representation in the video display unit and, if appropriate, it canintervene in the control of the vehicle through a control unit 90.Exemplary actuator systems include systems for influencing the steeringangle and intervening in the engine and brake control systems.Object-detection unit 30, in monitoring in the environment, does notinitially presuppose a preestablished parking geometry or the like, butrather, on the basis of the actually existing image or object-detectiondata, generates a description of the environment. The modeling of theenvironment and the images of the cameras are thus combined into onerepresentation by superimposition unit 40. This representation is usedto inform the driver comprehensively about the current situation in thevehicle environment. In this context, the object detection systemsupplies the location and the number of objects and, depending on thesensor system used, it can supply varying object sizes in varyingdegrees of precision. This data (size and distance of the objects) canalso be displayed in the video display unit, by object-detection unit 30also transmitting this data to superimposition unit 40 in theappropriate manner. Using maneuver calculating unit 50, the deviceaccording to the present invention, in addition to passively monitoringthe momentary situation in the vehicle environment, can also activelyassist the driver in controlling the vehicle. Object-detection unit 30transmits to maneuver calculating unit 50 the modeling data or theobject-detection data of the environment. For certain scenarios, thevehicle course is then calculated by maneuver calculating unit 50. Inwhat follows, a few advantageous possibilities are indicated:

1. The vehicle course yields to detected obstacles.

2. The vehicle course leads into a parking space parallel to theroadway.

3. The vehicle course leads into a parking space perpendicular to theroadway.

4. The vehicle course leads into a parking space diagonal to theroadway.

5. The vehicle course leads to a predetermined setpoint position betweena plurality of obstacles, it being possible, e.g., to configure thelatter. Thus, an example of a setpoint position is the position in thehome garage, and an example of an obstacle is the gate in front of thisgarage. For calculating the above-mentioned vehicle courses, it can alsobe taken into account that a trailer is attached to a vehicle and thatthe driver should initially countersteer in order to arrive at apredetermined setpoint position. Either the maneuver calculating unit isconfigured so that it automatically detects the above-mentioneddifferent situations, or the driver has the possibility of selecting thecorresponding parking variant in the selection means installed in thedashboard. Certain maneuvers, such as parking in the home garage orother standard maneuvers, can also be stored in the memory or bepreprogrammed. For this purpose, the maneuver calculating unit has asuitable program memory, from which the stored maneuvers can beretrieved.

From the above-mentioned considerations, according to an embodiment ofthe present invention, the following gradations for the degree ofvehicle intervention are provided:

1. The images of the video cameras are displayed in the video displayunit mounted in the dashboard or in the center console.

2. At the same time, object information, such as size, position, anddistance, are overlaid in the appropriate manner.

3. Additionally, information regarding the vehicle condition isoverlaid, such as the steering angle adopted, which determines theappropriate driving path, the vision angle of the vehicle with respectto the road (i.e., the angle of the normal through the automobilerelative to the road normal), etc.

4. The steering maneuvers are calculated by maneuver calculating unit 50and are displayed in the video display unit. The steering angle iscalculated by the system as a function of the current situation and isoverlaid on the camera image in addition to the current vehicle steeringangle. On the basis of the overlaid steering angle, the driver makes avariance comparison and turns the steering wheel accordingly. In thismanner, the driver continually maintains complete control. The necessarysteering direction can also (additionally) be indicated bycorrespondingly overlaid arrows.

5. The device automatically sets the calculated steering angle viacontrol unit 90, so that there is a direct intervention in the steering.However, the steering angle can at any time be overridden by the driver,and thus an individually appropriate steering angle can be chosen. Thedriver himself thereafter controls the vehicle longitudinal motion,i.e., the engine and brakes.

6. In a completely automatic operating mode, the device controls thevehicle entirely automatically by intervening both in the steering aswell as in the engine control on the basis of the calculated vehiclemaneuvering data. Here too, the maneuver can at any time be broken offby the driver.

In an alternative embodiment, Object-detection unit 30 and maneuvercalculating unit 50 can be configured so that the active maneuvercontrol is automatically broken off in accordance with Point 6, forexample, in the following situations:

1. Objects, especially rapidly moving objects, appear in the drivingarea, especially in the rear vehicle area.

2. Objects were surveyed in a grossly mistaken fashion.

3. There exists an acute danger of collision with an object.

4. Endangered objects (e.g., living things) appear.

FIG. 4 depicts an embodiment 55 of the video display unit for displayingthe images supplied by the cameras and processed by the control unit.Video display unit 55 has an image area 12 as well as both vertical edgeareas 13 as well as horizontal edge areas 14, which together form theborder of image area 12.

In image area 12 itself, and in the vertical as well as the horizontaledge areas of the image area, colored bars are displayed for opticallysignaling objects in the vehicle environment that are not covered by thevision field of video camera(s). Additionally, at a suitable location, apictogram 15 can be overlaid for representing the vehicle itself. In theenvironment of this vehicle, the position of detected objects can alsobe plotted. Video display unit 55 can advantageously be integrated inthe dashboard or in the center console of the vehicle.

As the warning color, a signal color such as red can be used in the edgeareas. Depending on which bar lights up, the driver knows immediately towhich side of the vehicle he should direct his attention although thecamera image still is not supplying any information to him. Additionalparallel support can be provided by acoustical display 6.

FIG. 5 depicts a further embodiment 5′ of the video display unit. Inimage area 12, a cube 115 is depicted, which in its size and shaperoughly represents a detected object. In area 116, data can be overlaidfor the distance or the size of this object. Lines 16 mark the drivingpath of the vehicle, if the given steering angle were to be maintained.Lines 17 mark the driving path which the vehicle would cover if thedriver were to follow the calculated route. Arrows 18, which light up,alternatively, depending on the direction in which the driver issupposed to turn the steering wheel, show him how he should steer inorder to achieve the steering angle suggestion that is indicated bylines 17.

All of the cited information is superimposed in image area 12 along withthe image of the video camera in order to provide rapid and preciseinformation to the driver concerning the situation. When the vehicleguidance is performed manually, it is also possible, if sensors andcameras are also present in the forward vehicle area, to provide anautomatic switchover between the front and the rear video cameradepending on which gear the driver has currently selected.

FIG. 6, by way of example, shows a parking maneuver of vehicle 7 behindparked vehicle 107 and in front of parked vehicle 108. The samereference numerals as in the previous Figures designate the samecorresponding parts.

In particular, laterally arranged object-detection sensors 9 a and 9 bmonitor areas that cannot be observed by fixedly-installed camera 8 a,but which are relevant for parking, particularly if carelesspedestrians, for example, enter into the areas just outside vision field8 a of camera 1 a. In this case, the driver receives through the videodisplay unit an appropriate warning and can act accordingly until thesepedestrians once again move out of the dead angle of the vehicle. Frontobject-detection sensors 9 c, 9 d, and 9 e also supply the precisedistance to forward vehicle 107 and make parking easier for the driverespecially when vehicle chassis are not completely visible, making theborders of the vehicle hard to determine from the driver's seat.

1. A device for monitoring the environment of a vehicle being parked,comprising: at least one video camera, the at least one camera having avision field the vision field being fixed relative to the vehicle; atleast one display unit, one of the at least one display unit being avideo display unit for representing the vision field; and at least oneobject-detection sensor for detecting objects in an area outside of anddirectly adjacent to the vision field; wherein a driver of the vehicleis informed via the at least one display unit of an existence of theobjects located outside of the vision field of the at least one videocamera detected by the at least one object-detection sensor.
 2. Thedevice as recited in claim 1, wherein the existence of the objectsoutside of the vision field are represented in edge areas of the videodisplay unit.
 3. The device as recited in claim 1, further comprising:an object-detection unit; and an image processing unit; wherein the atleast one object-detection sensor is coupled to the object-detectionunit, the object-detection unit being coupled to the image processingunit for selected digital image processing of video images from the atleast one video camera, objects in the vision field of the at least onevideo camera being automatically detected and communicated to thedriver.
 4. The device as recited in claim 3, further comprising: adownstream superimposition unit; wherein objects detected by theobject-detection unit are modeled using simple geometric forms, and thegeometric forms are superimposed on a video image using the downstreamsuperimposition unit.
 5. The device as recited in claim 1, furthercomprising: a maneuver calculating unit, the maneuver calculating unitprocessing external parameters including an instantaneous steeringangle; wherein an actual-steering-angle display can be carried out inthe video display unit by applying calculations made in the maneuvercalculating unit.
 6. The device as recited in claim 5, theobject-detection unit supplies the maneuver calculating unit with dataconcerning detected objects and the maneuver calculating unit calculatesa parking maneuver on the basis of the supplied data.
 7. The device asrecited in claim 6, wherein the parking maneuver is represented in thevideo display unit in the form of a steering angle suggestion.
 8. Thedevice as recited in claim 6, further comprising: a control unit, thecontrol unit being coupled to the maneuver calculating unit, the controlunit automatically executing the parking maneuver.
 9. The device asrecited in claim 8, wherein the maneuver calculating unit includes astorage unit for storing standard parking maneuvers including parking ina private garage, a standard parking maneuver being accessible forautomatically executing the parking maneuver when the correspondingenvironment is detected by the object-detection unit.
 10. The device asrecited in claim 1, wherein the at least one object-detection sensor isone of an ultrasound sensor, a radar sensor, and a lidar sensor.
 11. Thedevice as recited in claim 3, wherein distances to detected objects arecalculated using the object-detection unit, numerical values of thedistances being overlaid in the video display unit.